Method for estimating a quantity of a blood component in a fluid canister

ABSTRACT

A system and method for assessing the concentration of a fluid component within a container, the method comprising: receiving data associated with an image of the canister; from the image, detecting a color grid comprising color elements coupled to the canister,; selecting a region of the image corresponding to a portion of the canister; determining a match between a detected color of the region and a shade in the set of colors associated with the color grid captured in the image; based upon a position of a color element corresponding to the shade in the color grid, retrieving a concentration of the blood component associated with the shade of color.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application Ser.No. 61/980,024, filed on 15 Apr. 2014, U.S. Provisional Application Ser.No. 62/080,927, filed on 17 Nov. 2014, and U.S. Provisional ApplicationSer. No. 62/102,708, filed on 13 Jan. 2015, which are each incorporatedherein in its entirety by this reference.

TECHNICAL FIELD

This invention relates generally to the surgical field, and morespecifically to a new and useful method for estimating a quantity of ablood component in a canister for use in surgical practice.

BRIEF DESCRIPTION OF THE FIGURES

FIGS. 1A and 1B depict flowchart representations of an embodiment of amethod for estimating a quantity of a fluid component;

FIGS. 2A, 2B, and 2C are schematic representations of a color grid of anembodiment of a method and system for estimating a quantity of a fluidcomponent;

FIGS. 3A-3F depict variations of blocks in an embodiment of a method andsystem for estimating a quantity of a fluid component;

FIG. 4 depicts variations of fluid receivers used in adaptations of anembodiment of a method and system for estimating a quantity of a fluidcomponent;

FIG. 5 depicts a flowchart representation of a variation of a method andsystem for estimating a quantity of a fluid component;

FIGS. 6A-6C depict variations of a method for estimating a quantity of afluid component;

FIGS. 7A-7C depict variations of a method for estimating a quantity of afluid component;

FIG. 8 depicts additional blocks in an embodiment of a method forestimating a quantity of a fluid component; and

FIG. 9 depicts an example of a system for estimating a quantity of afluid component.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

The following description of preferred embodiments of the invention isnot intended to limit the invention to these preferred embodiments, butrather to enable any person skilled in the art to make and use thisinvention.

1. First Method

As shown in FIGS. 1A and 1B, a method 100 for estimating a quantity of ablood component in a canister comprises: from an image of a canister,detecting a color grid coupled to the canister S110, the color gridcomprising an array of color elements, wherein each color element isassociated with at least one of a set of colors in a red spectrum;selecting a region of the image corresponding to a portion of thecanister, based upon a position of the color grid relative to thecanister S120; determining a match between a detected color of theregion and a shade of red in the set of colors associated with the colorgrid captured in the image S130; based upon a position of a colorelement corresponding to the shade of red in the color grid, retrievinga concentration of the blood component associated with the shade of redS140; and associating the concentration of the blood component with avolume of fluid contained in the canister S180, thereby estimating thequantity of the blood component in the canister.

Generally, the method functions to implement machine vision to estimatethe content of a blood component within a fluid canister. In particular,the method 100 analyzes an image of a fluid canister to identify a colorgrid printed on, applied to, or otherwise coupled to the canister, tocolor match a particular shade in the color grid to a region of theimage corresponding to the canister, and to estimate a concentration ofa blood component (e.g., red blood cells, hemoglobin, free hemoglobin,etc.) within the canister based on a blood component concentrationempirically determined and associated with the particular shade. Themethod can further determine a total volume of fluid within the canisterand combine this volume with the estimated concentration of the bloodcomponent to calculate a total amount (e.g., mass, volume, or weight) ofthe blood component within the canister.

In a specific application, the method 100 can be used to estimate a massof hemoglobin within a fluid canister present in a clinical/medicalenvironment, wherein the fluid canister is used for blood salvage duringa procedure, and wherein fluid within the canister includes at leastsome blood (e.g., in addition to saline and/or other bodily fluids of apatient). In relation to this specific application and variationsthereof, the surface of the fluid canister, without implementation ofthe method 100, can be prone to specular reflections/glare andvariations in ambient light can lead to variations in color of fluidwithin the fluid canister. As such, the method 100 includesimplementation of steps and components that function to mitigatespecular reflections and glare, as well as steps and components thatmitigate effects of variations in ambient lighting.

The method 100 can therefore implement methods and techniques describedin U.S. application Ser. No. 13/544,646 entitled “System and Method forEstimating Extracorporeal Blood Volume in a Physical Sample” and filedon 9 Jul. 2012, U.S. application Ser. No. 13/894,054 entitled “Systemand Methods for Managing Blood Loss of a Patient” and filed on 14 May2013, U.S. application Ser. No. 13/738,919 entitled “System and Methodfor Estimating a Quantity of a Blood Component in a Fluid Canister” andfiled on 10 Jan. 2013, and U.S. application Ser. No. 14/072,625 entitled“Method for Triggering Blood Salvage” and filed on 5 Nov. 2013, whichare each incorporated herein in its entirety by this reference.

The blood component can be any of whole blood, red blood cells,hemoglobin, platelets, plasma, white blood cells, analytes, or any othersuitable blood component or combination of blood components.Furthermore, the blood component can comprise any component derived fromany of the above blood components (e.g., intracellular content,molecular content, etc.). The method can additionally or alternativelyimplement similarly techniques to estimate a concentration (and anamount) of a non-blood component within the canister, such as saline,ascites, bile, irrigant saliva, gastric fluid, mucus, pleural fluid,interstitial fluid, urine, fecal matter, or any other bodily fluid of apatient.

The fluid canister can be a suction canister implemented in a surgicalor other medical, clinical, or hospital setting to collect blood andother bodily fluids. For example, the canister can include a surgicalfluid canister defining a translucent polymer vessel including a seriesof fluid volume indicator markings (e.g., horizontal indicator markings)arranged vertically along a wall of the vessel and visible from outsidethe container. The canister can alternatively be a blood salvagecanister, an intravenous fluid bag, or any other suitable blood- orfluid-bearing container for collecting surgical waste or recoveringbiological fluid. The fluid canister is also transparent, translucent,or includes a transparent or translucent region along a wall (e.g.,vertical wall, slanted wall) of the container such that an image of thecanister can include sufficient information to enable the method 100 tocolor match fluid contained in the fluid canister to a color shadeprinted onto, applied onto, or otherwise associated with the canister,and to accordingly estimate a concentration of the blood componentwithin the canister.

The method 100 can therefore be useful in quantifying an amount and/or aconcentration of a blood component (e.g., hemoglobin) and/or otherfluids (e.g., saline) contained within a fluid canister throughnon-contact means and in real-time, such as during a surgery or othermedical event. A patient's blood loss and euvolemia status can then betracked according to these data, such as described in U.S. patentapplication Ser. No. 14/072,625. However, the method 100 can beapplicable in any other scenario or environment to estimate aconcentration and/or amount of a blood component or other fluid orparticulate in a vessel.

The method 100 can thus be implemented by a computer system as a fluidcanister analyzer that analyzes a photographic image (e.g., digitalimage) of a canister to estimate the quality of a fluid containedtherein. The computer system can be cloud-based (e.g., Amazon EC2), amainframe computer system, a grid-computer system, or any other suitablecomputer system. For example, the method 100 can be implemented by ahandheld (e.g., mobile) computing device, such a smartphone, a digitalmusic player, or a tablet computer executing a native blood componentanalysis application, such as shown in FIG. IA. For example, an imageacquisition module integral with the computing device can capture theimage of the fluid canister, and a processor integral with the computingdevice can implement Blocks of the method 100 to extract informationindicative of the quality of the fluid in the canister from the image.The computing device can additionally or alternatively communicate witha remote server, such as over the Internet via a wireless connection,the server can perform one or more Blocks of the method, 100 and one ormore outputs of the method 100 can be transmitted from the remote serverback to the computing device for further analysis and/or subsequentpresentation to an entity (e.g., a nurse, an anesthesiologist). Thecomputing device can also include or can be coupled to a digitaldisplay, and the method 100 can present information to the entitythrough the display.

Alternatively, the method 100 can be implemented as a standalone bloodvolume estimation system including a fluid canister, a fluid canisterstand, an image acquisition module, a camera stand configured to supporta camera of the image acquisition module adjacent the fluid canister, adigital display, a processor configured to perform at least a portion ofthe method, and/or a communication module configured to communicate witha remote server that performs one or more Blocks of the method 100. Inthis implementation, the camera can be substantially non-transientlypositioned relative to a fluid canister stand such that the cameraremains in a suitable position to capture an image of a canistersubstantially throughout a surgery or other medical event and/or untilthe canister is full. The blood volume estimation system can thusregularly capture and analyze images of the fluid canister, such asevery thirty seconds or every two minutes. The blood volume estimationsystem can further communicate (e.g., via Bluetooth) with another one ormore systems implementing any of the methods described in U.S.application Ser. Nos. 13/544,646, 13/894,054, 13/738,919, and 14/072,625to form a fluid management system for generating a substantiallycomprehensive analysis of one or more of: extracorporeal blood volume,total patient blood loss, patient euvolemia status, and any othersuitable patient state in a clinical or non-clinical environment.However, the method 100 can be implemented in or by any other suitablecomputer system, computing device, or combination thereof.

Furthermore, variations of the method 100 and system can be adapted toprocess image data (or other data) derived from any other suitable fluidreceiver (e.g., canister, test strip, absorbent pad, surgical textile,sponge, fluid receiving bag, drape, cell salvage system, drain device,etc.) associated with or otherwise coupled to a color grid (e.g.,incorporated into a quick response code, incorporated into a barcode,incorporated into a rectilinear array, incorporated into an axiallysymmetric array, etc.), wherein the fluid receiver is configured toreceive (e.g., receive into a cavity, receive upon absorption) a volumeof fluid (e.g., urine, saline, ascites, bile, irrigant saliva, gastricfluid, mucus, pleural fluid, interstitial fluid, fecal matter,non-biological fluid, etc.). As such, variations of the method 100 andcolor grid 300 described below can facilitate mitigation of ambientlight effects in an environment of the fluid receiver, in determining aconcentration and/or an amount of a fluid component within a volume offluid received at the fluid receiver.

2. Color Grid, Canister, and Processing Module

As shown in FIG. 2A, in one embodiment, the color grid 300 includes anarray of color elements 310 (e.g., regions comprising one or more“pixels”) of distinct colors, wherein the array of color elements isprinted, applied, projected onto, or otherwise coupled to an exteriorsurface of the canister. In one example, the color grid 300 is appliedin the form of a decal 301 onto the canister 200. In this example, thedecal 301 can include: an adhesive layer 320; an opaque layer 325arranged over a first portion 321 of the adhesive layer 320; an array ofcolor elements 310, each of a discrete shade of red, arranged over theopaque layer 325; and an anti-glare layer 330 laminated over the arrayof color elements 310 and directly over a second portion 322 of theadhesive layer 320 adjacent the first portion 321. The adhesive layer320 is preferably substantially transparent in order to allow a color offluid within the canister 200 to be accurately represented in image dataof the canister-color grid assembly; however, the adhesive layer 320 canalternatively have any characteristic color (e.g., shade, hue,saturation, intensity, opacity, etc.) in order to facilitate imageprocessing (e.g., color correction) of an image including the color grid300.

In this example, the color grid 300 can then be applied (i.e., “stuck”)over an exterior surface of the canister 200 by a manufacturer beforeshipping the canister to an end-user, or can alternatively be coupled toa canister by a nurse, anesthesiologist, or other end-user before orduring a surgery or other medical event. However, variations of thespecific example of the color grid 300 can omit one or more of: theadhesive layer 320, the opaque layer 325, and the anti-glare layer 330(e.g., in relation to a fluid receiver that has a matte finish orotherwise does not produce glare), and can additionally or alternativelyinclude any other suitable layers that enhance or otherwise facilitateimage processing of an image of the color grid within a field of view.

The opaque layer 325 of the color grid 300 functions to provide arepeatable background for the color grid 300, such that perceived (orimaged) colors of shades in the color grid are not substantiallyaffected by lighting conditions, reflections of other onto the colorgrid 300 or canister, or an opacity of fluid contained within anadjacent canister (or other object). In specific examples, the opaquelayer 325 can include one or more of: an aluminum sheet, a foil sheet, awhite-enameled copper sheet, and any other suitable material thatprovides an opaque region for the color grid 300.

The anti-glare layer 330 of the color grid 300 functions to reduce orsubstantially eliminate glare over the color grid 300 and an adjacentregion of fluid in an image of the canister 200 such that the imagecontains sufficient color data to enable the method 100 to correlate acolor of the fluid in the image with a blood component concentration (orsimilarly to enable the variations of the method 100 described below tocolor normalize the image and to correlate a color of the fluid in theimage with a concentration of the blood component). The anti-glare layer330 can further function to maintain a consistent surface treatmentacross both the color grid and a translucent or transparent surface ofcanister 200, such that a portion of an image of the canister 200corresponding to fluid contained therein can be color normalized and/ormatched to a particular shade in the color grid 300 to estimate aconcentration of the blood component within the canister 200. In onespecific example, the anti-glare layer 330 can extend beyond a portionof the color grid 300 comprising the array of color elements 310, thusproducing a region 6 of the fluid canister 200, immediately adjacent tothe array of color elements, that is coupled to the anti-glare layer330. The region 6 can thus be used, for instance, as in Block S120 ofthe method 100, for analysis and eventual estimation of the amount ofthe blood component present within the fluid canister 200. The region 6can, however, be defined relative to a position of the color grid 300 onthe canister 200 in any other suitable manner.

In a few such alternative variations, the color grid 300 can be arrangedon a placard adjacent the canister 200, on a work (e.g., table) surfaceadjacent the canister 200, on a canister 200 stand supporting thecanister 200, on a surgical glove near or contacting the canister 200,or on any other surface near the canister 200 such that the color grid300 and the canister 200 can be imaged together.

The array of color elements 310 of the color grid 300 includes multiplevarieties of a color (i.e., hue characterized by a wavelength of light)that is characteristic of the blood component (i.e., red), wherein thevarieties of the color are differentiated according to one or more ofvalue/brightness (e.g., in terms of shade or tint) andsaturation/chroma. Variations of the array of color elements 310 can,however, include multiple hues of color. Furthermore, the colors of thearray of color elements 310 of the color grid 300 can be described inRGB triplet format (e.g., in terms of intensities of red, green, andblue components), hexadecimal format (i.e., hex format), and/or anyother suitable format that facilitates downstream processing ofinformation derived from the array of color elements 310 at a computingsystem. The colors of the array of color elements 310 can further existwithin any suitable color space (e.g., Adobe RGB color space, CIE 1931XYZ color space, CIE L*a*b color space, etc.) in relation to printing ofthe color grid 300, projection of colors of the color grid 300,perception of colors of the color grid 300, and/or any other suitablefactor.

In one variation, the array of color elements 310 of the color grid 300can include a set of red elements spanning hex color values in the#xx0000 range (e.g., from #200000 to #FF0000) or spanning RGB colorvalues in the rgb(x,0,0) range (e.g., from (32,0,0) to (255,0,0)).However, in this variation, the color grid 300 can additionally oralternatively represent any other suitable color (e.g., in terms of hue,saturation/chroma, brightness/value) that may be substantially near acolor of a bloodied (i.e., blood-containing) fluid. In the example shownin FIG. 2A, each color in the color grid 300 can be represented as anelement arranged in a grid format wherein each element is ofsubstantially uniform shade (i.e., having a low degree of variabilityacross pixels of the color element).

In the example shown in FIG. 2A, the color grid 300 includes arectilinear grid layout of elements in which each element represents aunique color brightness (i.e., shade or tint) of a color (e.g., red)within the set of colors. In this example, the elements can be arrangedwith substantially similar color nearby to yield a relatively smoothcolor transition across the color grid 300 in one or more lineardirections. In another example, the color grid can include multipleelements representing the same color shade. In this example, the arrayof color elements 310 can be arranged with colors having specificcharacteristics (e.g., brightness, saturation, etc.) forming a patternacross the color grid 300. In more detail, a set of elements of the same(dark) shade of red can form a cross centered in a square color grid 300and split the color grid 300 into four quadrants, and additionalelements can fill the first quadrant with increasingly lighter shades ofred nearing the far corner of the first quadrant. In this example, thefirst quadrant can be mirrored vertically, diagonally, and horizontallyinto the second, third, and fourth quadrants of the color grid,respectively, thus allowing the color grid 300 to be less affected byorientation (e.g., in relation to 90° rotations of the color grid). In avariation of this example, the color grid 300 can be axially symmetric(e.g., using a circular or otherwise axially symmetric array of colorelements), thus allowing the color grid 300 to be even less affected byorientation (e.g., of the canister, of the color grid relative to thecanister). The color grid 300 can, however, comprise or omit anysuitable axis of symmetry.

In an implementation similar to the above example, with a rectilineararray of color elements 310, the color grid 300 contains colored blocksof various discrete colors (e.g., red hues, saturation of red hues,intensity of red hues, etc.) arranged in a matrix barcode 5 (e.g., aQuick Response or “QR” code, another optical machine-readable barcode,etc.), as shown in FIG. 2B. In this implementation, the set of coloredblocks—along with a set of white blocks (e.g., interspersed whiteblocks)—can be arranged in a square grid pattern printed on or otherwiseapplied to a sticker (or decal, etc.), wherein information including oneor more of: alignment information, position information, versioninformation, identification information, and any other suitableinformation is encoded within the square grid pattern based on the(relative) positions of colored and white blocks therein. A device canthus implement machine vision techniques according to blocks of themethod 100 described herein to account for distortion of the matrixbarcode 5 (e.g., due to a non-planar surface of the canister 200), toextract color information from color elements of the color grid 300 forapplication to a color of fluid within the images canister on which thesticker (or decal) is applied (or printed); the device can also decodethe grid pattern of the color grid to retrieve additional informationrelated to the color grid and/or to the canister. In thisimplementation, information accessible upon decoding of the matrixbarcode 5 can be at least partially redundant with other informationprinted or otherwise provided (e.g., printed) on a surface of the colorgrid 300. For instance, the color grid 300 can include a region withprinted text describing an alphanumeric identifier of the fluid canister200, and the alphanumeric identification can also be accessible upondecoding of the matrix barcode 5. Thus, in relation to a set ofcanisters (i.e., for multiple patients, for a single patient within amedical environment), a color grid 300 can be used to uniquely identifya canister, while still including a set of color elements (i.e., inunique configurations) that is substantially similar across the set ofcanisters.

In a first example that implements a color grid 300 with encodedinformation, a scanning device of the mobile computing device (e.g.,optical sensor module of a mobile computing device) can be configured tolocate one or more positional features (e.g., QR-code corner features)of the color grid 300, to extract an identifier (e.g., alphanumericidentifier) of the canister 200 from the matrix barcode 5, and totransmit the identifier and the image data of the canister to aprocessing module (e.g., computing system, server) in communication withthe mobile computing device (or other image acquisition device). Theprocessing module can then use the identifier to determine one or morecharacteristics of the canister (e.g., morphological characteristics,make, model, serial number, geometry, maximum internal volume, wallthickness, and/or surface glossiness, etc.) and to reconstruct atemplate of how the color grid would look (i.e., without distortion) ina known configuration (e.g., canonical space, with a certainorientation, distance, and perspective relative to an optical sensor).In particular, the template of how the color grid looks in canonicalform can be derived from an electronic file used to print the colorgrid, wherein the electronic file represents the configuration of thecolor grid without any warping or distortion, and wherein the electronicfile is retrieved by the processing system upon reception of theidentifier. The processing module can then be configured to identify anundistorted configuration of the color grid 300 based upon positionaladjustment of the one or more positional features of the color grid 300relative to the template, as shown in FIG. 3A.

In the first example, the processing module can comprise a paletteextraction module configured to use the identifier (e.g., alphanumericidentifier) of the canister to define masks of a set of regions of thearray of color elements 310 of the color grid 300 in canonical space,and to use the one or more positional features (e.g., extracted QR-codecorners) to fit a transformation model (e.g., homography) betweencanonical space and image space associated with the image data. Thetransformation model can then be used, by the processing system, totransform each of the set of regions into image space. Then, for each ofthe set of regions, a set of pixels of an associated transformed maskcan be processed to determine an average (e.g., median, mean) colorvalue (i.e., RGB color value, hex color value) as representative of theassociated region, as shown in FIG. 3B. As such, the processing modulecan extract color values of the array of color elements 310 of the colorgrid 300 in a manner that is consistent across all color grids for a setof canisters. In the first example, the set of regions comprises 9regions, each representing a distinct color value; however, invariations of the specific example, the set of regions can alternativelycomprise any other suitable number of regions for normalization of theimage (e.g., in relation to ambient light conditions) and/or extractionof blood component information from the image.

Then, in the first example, the processing module can be configured toidentify a region of the canister 200 for determination of the amount ofthe blood component (e.g., hemoglobin mass), wherein the region iscaptured in the image data and associated with a position of the colorgrid 300 on the canister 200. The processing module can then beconfigured to use the transformation model (e.g., homography) totransform the region of the canister 200 from canonical space into imagespace, and to inscribe a bounding region inside the region, as shown inFIG. 3C. In the first example, the processing system can then beconfigured to normalize the image data to account for effects of ambientlight based upon a reference palette of the array of color elements 310,wherein the reference palette includes a representation of the array ofcolor elements 310 in a specific configuration (e.g., derived from areference image with controlled light conditions, derived from colorinformation used to print the array of color elements, etc.). In thefirst example, the reference palette can be used to normalize the set ofregions of the color palette output by the palette extraction moduleaccording to a set of fit functions that transform red channel, greenchannel, and blue channel color information values captured in the setof regions of the color palette into normalized red channel, greenchannel, and blue channel values, thereby outputting a normalized colorpalette that is normalized for ambient light conditions, as shown inFIG. 3D.

In the first example, the processing system is then configured to removenoise or other artifacts (e.g., artifacts caused by air bubbles underthe color grid 300, artifacts caused by debris between the color grid300 and the canister, etc.) from the image data of the region ofinterest for blood component analysis, wherein in a specific example,artifact removal is performed according to a maximally stable extremalregions (MSER) algorithm to determine an initial mask of substantiallyartifact-less subregions of the region of interest, as shown in FIG. 3E.Then, the processing system is configured to remove any pixels whosecolor value is significantly different from a median color value ofpixels of the initial mask, in order to remove aberrations present afterimplementation oft the MSER algorithm. The processing system canadditionally or alternatively implement a Laplacian of Gaussianalgorithm, a difference of Gaussians algorithm, and/or a determinant ofHessian algorithm to remove noise or other artifacts.

Finally, the processing system is configured to determine an estimatedhemoglobin mass from the region of interest of the canister, based upona parametric model. In particular, the parametric model implements asupport vector machine (SVM) algorithm with a radial basis function(RBF) kernel that generates a hemoglobin concentration derived from redvalue, green value, and blue value color intensities the region ofinterest, and multiplies the hemoglobin concentration by the volume offluid within the canister can to determine the estimated hemoglobinmass. Additionally or alternatively, as shown in FIG. 3F, any othersuitable parametric model (e.g., linear regression model, power curvedriven regression model, other regression model, etc.) or anon-parametric model can be implemented by the processing system todetermine an amount of any other suitable blood component within thecanister (or other fluid receiver), as described in one or more of: U.S.application Ser. No. 13/544,646, U.S. application Ser. No. 13/894,054,U.S. application Ser. No. 13/738,919, and U.S. application Ser. No.14/072,625.

Additionally or alternatively, variations of the above system and methodcan incorporate multiple color grids coupled to a canister (or otherfluid receiver), in order to enable correction of location-dependentambient lighting effects, in characterizing fluid at the canister (orother fluid receiver). For instance, a first color grid positioned at afirst location on the canister and a second color grid positioned at asecond location on the canister can be used to correct effects caused bydifferences in the way ambient light hits the first location and thesecond location of the canister.

As shown in FIG. 4, variations of the above example of a system can alsobe adapted to process image data (or other data) derived from any othersuitable fluid receiver (e.g., canister, test strip, absorbent pad,surgical textile, sponge, fluid receiving bag, drape, cell salvagesystem, drain device, etc.) associated with or otherwise coupled to acolor grid (e.g., incorporated into a matrix barcode, incorporated intoa quick response code, incorporated into a barcode, incorporated into arectilinear array, incorporated into an axially symmetric array, etc.),wherein the fluid receiver is configured to receive (e.g., receive intoa cavity, receive upon absorption) a volume of fluid (e.g., urine,saline, ascites, bile, irrigant saliva, gastric fluid, mucus, pleuralfluid, interstitial fluid, fecal matter, etc.). As such, variations ofthe system above can facilitate mitigation of ambient light effects inan environment of the fluid receiver, in determining a concentrationand/or an amount of a fluid component within a volume of fluid receivedat the fluid receiver.

In another example that implements a color grid 300 with encodedinformation, a scanning device can extract manufacturer, supplier, batchnumber, serial number, version, hospital, a uniform resource locator(“URL”) or other web address, and/or any other suitableinformation—pertaining to the canister 200 and/or to the color grid300—from the matrix barcode 5 by: identifying positions of white (orother light-colored) blocks and positions of dark (e.g., red) blocks inan image of the matrix barcode 5 captured by the device; identifyingposition markers (e.g., QR-code corners) within the matrix barcode 5;determining an orientation of the matrix barcode based on relativelocations of the position markers; identifying a set of modules withinthe matrix barcode based on the orientation of the matrix barcode 5;decoding each module into a character based on the orientation of thematrix barcode and positions of light blocks and dark (e.g., red) blockswithin each module; and assembling characters extracted from the setmodules into a meaningful resource. In this example, the device canassemble the characters into a URL and retrieve data related to thecanister and/or the color grid from the URL via a network (e.g.,Internet) connection, such as make, model, serial number, geometry,maximum internal volume, wall thickness, and/or surface glossiness, etc.of the canister onto which the sticker (with the matrix barcode 5) isapplied. The device can additionally or alternatively access the URLextracted from the matrix barcode 5 to retrieve color template data forthe matrix barcode 5, such as hemoglobin concentration, red blood cellconcentration, and/or hemolysis proportion, etc. corresponding to eachcolor represented by colored blocks in the color grid; the device canthen implement template matching techniques described herein to match acolor of a portion the image corresponding to fluid in the canister to aparticular color represented in the color grid shown in the same imageand apply hemoglobin concentration, red blood cell concentration, and/orhemolysis proportion, etc. corresponding to the particular color—asspecified in the color template data retrieved from the URL—to anestimate for qualities of fluid contained in the canister, as describedbelow. However, the color grid can include any other number of shades ofred (or other hue) arranged in any other suitable way.

In the foregoing examples, each particular color value represented inthe color grid 300 can be used to facilitate image normalization inconsideration of ambient light effects, and/or be paired with a specificblood component concentration. For example, fluid solutions containingsaline and 0.0%, 2.5%, 5.0%, 7.5% . . . 92.5%, 95.0% red blood cells byvolume can be prepared in various fluid containers, each fluid containerincluding a substantially identical color grid with an array of elementsof various shades of red. In this example, an image of each fluidsolution can be imaged, and, for each fluid solution, a color of thefluid solution extracted from the corresponding image and the color ofthe solution matched to one color element (or to two nearest colorelements) in the color grid. Each color value represented in the colorgrid 300 can thus be matched to and associated with a particular redblood cell concentration in saline. Thus, during a subsequent surgery orother medical event, the method can reverse this procedure to extract acolor from a region of an image of a fluid canister corresponding tobloodied fluid, to match this color to a color element in the color gridapplied to the canister, and to estimate a red blood cell concentrationof the fluid within the canister based on the red blood cellconcentration associated with the selected color element.

In another implementation, the color grid includes elements representingmultiple pure colors, such as red, blue, and green, such as shown inFIG. 2C. In this implementation, the method can color normalize an imageof the canister 200 by adjusting a red, blue, green, contrast,brightness, and/or other color or image parameter to match portions ofan image corresponding to the color elements to a know color value ofeach of these elements (e.g., rgb(256,0,0), rgb(0,0,256), and rgb(0,256,0)), as described below. However, the color grid 300 can include anyother number of shades of any other color arranged in any other suitableway.

The color grid 300 can also include one or more reference points (e.g.,positional features) of known size, geometry, and/or spacing, and one ormore blocks of the method 100 can implement machine vision techniques toidentify the reference point(s) in an the image of the canister and toscale, rotate, translate, skew, and/or transform the image to match theidentified reference point(s) in the image to its known (i.e., true)size, geometry, and/or spacing. For example, the color grid 300 caninclude one or more printed symbols, such as a series of corner or “+”symbols arranged about the colored elements. Alternatively, the elementscan be of a specific size and geometry (e.g., 3 mm×3 mm squareelements), which one or more Blocks of the method 100 can identify in animage of the canister and implement to transform the image, to extract adimension from the image, and/or to locate a point of interest (e.g., aparticular color element in the color grid) in the image.

3. Template Matching

Block S110 recites: from an image of a canister, detecting a color gridcoupled to the canister, the color grid comprising an array of colorelements, wherein each color element is associated with at least one ofa set of colors in a red spectrum, and Block S120 of the method recitesselecting a region of the image corresponding to a portion of thecanister, based upon a position of the color grid relative to thecanister.

Generally, once an image of the canister is captured, (e.g., asdescribed in U.S. patent application Ser. No. 13/738,919), Block S110can comprise implementing machine vision techniques to identify thecolor grid in the image, and/or one or more positional features (e.g.,QR-code corners) associated with the color grid in the image, examplesand variations of which are described above in relation to theprocessing system. Block S110 can comprise implementing one or more of:object localization, segmentation (e.g. edge detection, backgroundsubtraction, grab-cut-based algorithms, etc.), gauging, clustering,pattern recognition, template matching, feature extraction, descriptorextraction (e.g. extraction of texton maps, color histograms, HOG, SIFT,etc.), feature dimensionality reduction (e.g. PCA, K-Means, lineardiscriminant analysis, etc.), feature selection, thresholding,positioning, color analysis, parametric regression, non-parametricregression, unsupervised or semi-supervised parametric or non-parametricregression, and any other type of machine learning or machine vision toidentify the color grid (or a portion of or a symbol on the color grid)in the image.

Block S110 can also comprise implementing machine vision techniques toidentify each discrete color element of an array of color elements inthe color grid in the image and to extract a color characteristic foreach identified color element. For example, for one identified examplecolor element, Block S110 can comprise determining an average colorvalue (e.g., a hex color value, an rgb( ) color value) of each pixelcorresponding to one color element in the image. In one variation, BlockS110 can comprise defining masks of a set of regions, each regionassociated with a color element of the array of color elements incanonical space, and using one or more positional features (e.g.,extracted QR-code corners) to fit a transformation model (e.g.,homography) between canonical space and image space associated with theimage. Block S110 can then comprise using the transformation model totransform each of the set of regions into image space and then, for eachof the set of regions, processing a set of pixels of an associatedtransformed mask to determine an average (e.g., median, mean) colorvalue (i.e. RGB color value, hex color value) as representative of thecolor element. Block S110 can then comprise storing these values withthe image, such as in the form of virtual overlay of color values overcorresponding color elements in the color grid in the image. As such,Block S110 can enable extraction of color values of the array of colorelements of a color grid in a manner that is consistent across all colorgrids for a set of canisters. Variations and examples of color elementidentification are described above in relation to the palette extractionmodule of the processing system described above.

Block S110 can further comprise assigning an address to each colorelement in the color grid in the image, such as in the form of a (m, n)matrix coordinate system for a the color grid that includes arectilinear array of color elements. For example, in a 10×10 color gridarray with one hundred color elements, Block S110 can address a top-leftelement at (1,1), an element immediately to the right of the top-leftelement (2,1), and a bottom right element (100, 100). Block S110 canalso comprise determining an orientation of the color grid in the image,such as based upon one or more positional features (e.g., associatedwith a position of a make, model, or serial number printed on thedecal), and assigning addresses to the color elements in the imageaccordingly. Block S110 can then group each determined color value of acolor element in the image with a corresponding color grid address, suchas in the form ((m,n),#XXXXXX).

With the color grid thus identified in the image, Block S120 cancomprise selecting a particular region of the image to match to a colorin the color grid. For example, as described above, the color grid canbe incorporated into a decal including a transparent region with ananti-glare surface adjacent the color grid, Block S110 can then compriseanalyzing the image to determine an orientation of the color grid on thecanister, and Block S120 can comprise selecting a region of the imageadjacent a particular edge of the color grid and corresponding to thetransparent region of the decal based on the determined orientation (andsize) of the color grid in the image. In this example, the decal can berectangular, and the color grid can be square and arranged over one halfand on one side of the decal, Block S110 can comprise detecting alongitudinal axis and a lateral axis of the decal, and Block S120 cancomprise selecting a group or cluster of pixels—in the image—groupedaround the longitudinal axis of the decal on a half of the decalopposite the color grid. Alternatively, the decal can define anelongated anti-glare strip arranged vertically along a side of thecanister, and Block S120 can comprise identifying a surface of fluid inthe canister (as described in U.S. application Ser. No. 14/072,625and/or U.S. application Ser. No. 13/738,919) and select a linear groupof pixels in the image corresponding to the anti-glare surface, distinctfrom the color grid applied to the canister, and running from a lowestpoint on the decal up to the identified surface of the fluid. However,Block S120 can comprise selecting any other region of the image in anyother suitable way.

Block S130 recites: determining a match between a detected color of theregion and a shade of red in the set of colors associated with the colorgrid captured in the image. Generally, Block S130 implements machinevision techniques to match a color of the particular region selected inBlock S120 to a color of a particular color element in the color gridcoupled to the canister and captured in the image.

In one implementation, Block S130 includes calculating a mean colorvalue (e.g., a hex color value, an RGB color value) from a color valueof each pixel in the region of the image selected in Block S120. BlockS130 can alternatively comprise calculating a color value for each pixelor for each subgroup of pixels in the region selected in Block S120 canthen identify a median color value in the selected region. However,Block S130 can comprise calculating and/or assigning a particular colorvalue to the region of the image corresponding to fluid in the canisterin any other suitable manner. Once the region of the image containingdata corresponding to fluid within the canister is thus quantified,Block S130 can comprise comparing the color value of this region todetermined color values of the color elements in the color grid in theimage. In particular, Block S130 can comprise comparing the color valueof the region to color values calculated for the color elements in BlockS110, and selecting a particular color element of color value nearestthat of the selected region. Alternatively, Block S130 can compriseselecting two color elements corresponding to two color values nearestthe color value of the selected region.

Block S130 can then comprise passing this nearest color value or thesetwo nearest color values—or one or more color grid address(es)associated with the nearest color value(s)—to Block S140 and/or BlockS150 of the method 100. Thus, though color information captured in oneimage of the canister and the color grid may differ from colorinformation captured in one another of the canister and the color grid(and from an image of another canister and another color grid), BlockS130 can enable matching of two (or more) colors represented in theimage. In particular, because the two matched colors were captured underthe same lighting, shadow, ISO, shutter speed, sampling rate aperture,and other conditions affecting recordation of light (i.e., color), thiscolor match may be inherently normalized for all such lighting andimaging conditions.

Block S140 recites: based upon a position of a color elementcorresponding to the shade of red in the color grid, retrieving aconcentration of the blood component associated with the shade of red.Generally, Block S140 functions to assign known concentrations of theblood component to particular color elements in the color grid in theimage. In one implementation, as described above, Block S110 identifiesa particular type, make, or model of the color grid on the imagedcanister by implementing optical character recognition (OCR) (or anyother machine vision technique) to read a QR-code, SKU number, a barcode, or a model number, etc. of the color grid from the image, andBlock S140 comprises passing this identifier to a remote color griddatabase. In this implementation, Block S140 receives a set of colorgrid addresses with corresponding known (i.e., empirically-determined)blood component concentrations and then maps the known blood componentconcentrations to respective color elements in the color grid based onthe addresses assigned to the color elements and the addresses receivedfrom the color grid database. For example, Block S140 can append theelement address and color value coordinates to read in the form((m,n),#XXXXXX , y % V). Thus in the example, one dark reddish-brownelement in the color grid can be assigned the coordinate ((5,6),#200000,67.5% V) and a light washed-out pink element in the color grid can beassigned the coordinate ((2,1),#FFCCCC, 5.0% V). Block S140 can thuscooperate with Block S110 to compile a list of coordinates, eachspecifying a location of an element in the color grid, a color value ofthe element in the image (which may differ amongst various images of thesame canister), and a corresponding blood component concentration.

In another implementation, Block S110 comprises extracting an identifierof the color grid in the image, Block S140 comprises selecting acomputer file of model color element addresses and known blood componentconcentrations and extracting a single blood component concentrationcorresponding to the (real) color element address selected in BlockS130. Similarly, for two color element addresses received from BlockS130, Block S140 can comprise selecting two corresponding bloodcomponent concentrations and then passing these values to Block S150.However, Block S140 can function in any other way to collect known bloodcomponent concentration data corresponding to one or more color elementsin the color grid.

Block S180 recites: associating the concentration of the blood componentwith a volume of blood contained in the canister, thereby estimating thequantity of the blood component in the canister. Generally, Block S180functions to assign a known concentration of the blood componentassociated with a particular color element to the fluid in the canister,based upon a color match between a corresponding color element and acolor of the fluid. In one implementation, Block S180 comprises applyingthe matched color value or the selected color element address to thelist of coordinates that specify color element locations, color values,and corresponding blood component concentrations, and outputting asingle blood component concentration for the fluid contained in thecanister shown in the image. In another implementation, Block S180comprises receiving two blood component concentrations—corresponding totwo color elements in the color grid nearest the fluid color shown inthe image—from Block S140 and averaging these blood componentconcentrations to generate a final estimate of the blood componentconcentration of fluid in the imaged canister. In a similarimplementation, Block S180 can comprise comparing a distance (e.g., in ahex color system) between the (average) color of the region of imageselected in Block S120 (and corresponding to fluid in the image) and thecolors of the nearest color elements of the color grid selected in BlockS140. In this implementation, Block S180 can then comprise applyingthese color distances to the color values to interpolate the bloodcomponent concentration in the canister. For example, Block S130 canextract color value rgb(130,2,2) from the region selected in Block S120and cooperate with Block S140 to select color element addresses ((8,4),rgb(128,0,3), 47.5% V) and ((7,5), rgb(136,0,1), 45.0% V) as nearest theselected region color. In this example, Block S180 can thus implementlinear interpolation to estimate that the actual red blood cellconcentration in the canister is approximately 45.6% by volume. However,Block S180 can comprise estimating the concentration of red bloods cellsor any other blood component in the canister in any other suitablemanner.

Block S180 can then comprise displaying this estimated concentrationvalue, such as on a display or other user interface arranged within theoperating room, as described in U.S. patent application Ser. No.14/072,625, and/or pass this concentration value to Block S192 forestimation of an amount (e.g., a weight, a volume, a mass) of the bloodcomponent in the canister based upon a volume of fluid within thecanister, as described further below.

Furthermore, as noted above, variations of the method 100 can be adaptedto process image data (or other data) derived from any other suitablefluid receiver (e.g., canister, test strip, absorbent pad, surgicaltextile, sponge, fluid receiving bag, drape, cell salvage system, draindevice, etc.) associated with or otherwise coupled to a color grid(e.g., incorporated into a quick response code, incorporated into abarcode, incorporated into a rectilinear array, incorporated into anaxially symmetric array, etc.), as shown in FIG. 4, wherein the fluidreceiver is configured to receive (e.g., receive into a cavity, receiveupon absorption) a volume of fluid (e.g., urine, saline, ascites, bile,irrigant saliva, gastric fluid, mucus, pleural fluid, interstitialfluid, fecal matter, etc.). In particular, color elements of the colorgrid, as described above, can comprise any other suitable (e.g.,non-red) color(s) configured to facilitate image data normalization forenvironmental condition effects, and/or for extraction of relevant fluidcomponent characteristics (e.g., concentrations, purities, etc.) from avolume of fluid at the fluid receiver.

In one such variation, as shown in FIG. 5, a method 100 b can comprise:from an image of a fluid receiver, detecting a color grid in proximityto the fluid receiver Snob, the color grid comprising an array of colorelements, wherein each color element is associated with at least one ofa set of colors; selecting a region of the image corresponding to aportion of the fluid receiver, based upon a position of the color gridrelative to the fluid receiver S120 b; determining a match between adetected color of the region and a color in the set of colors associatedwith the color grid captured in the image S130 b; based upon a positionof a color element corresponding to the color in the color grid,retrieving a concentration of a fluid component associated with thecolor S140 b; and associating the concentration of the fluid componentwith a volume of fluid contained in the canister S180, therebyestimating the quantity of the fluid component in the canister.

As such, variations of the method 100 can facilitate mitigation ofambient light effects (and other effects) in an environment of the fluidreceiver, in determining a concentration and/or an amount of a fluidcomponent within a volume of fluid received at the fluid receiver.

4. First Variation—Image Normalization and Concentration Extraction

As shown in FIGS. 6A and 6B, a first variation of the method 100′includes: within an original image of a canister, detecting a color gridcoupled to the canister S110′, the color grid comprising an array ofcolor elements, wherein each color element is associated with at leastone of a set of colors in a red spectrum; selecting a region of theoriginal image corresponding to a portion of the canister, based upon aposition of the color grid relative to the canister S120′; determining amatch between a detected color of the region and a shade of red in theset of colors associated with the color grid captured in the originalimage S130′; retrieving a true color characteristic corresponding to theshade of red in the original image based upon a position of the shade ofred in the color grid in the original image S150′; generating anadjusted image derived from the original image upon adjusting a colorsetting of the original image to align a color characteristic of theshade of red in the adjusted image to the true color characteristicS160′; extracting a redness value from a region of the adjusted imageS170′; and correlating the redness value with a concentration of theblood component within a volume of fluid contained in the canisterS190′.

Generally, this variation of the method 100′ functions to match a colorin a region of the image corresponding to fluid in the canister, to aparticular color represented in one or more elements in the color grid.This variation of the method 100 further functions to normalize theimage by adjusting at least one of: a red component (e.g., intensity), ablue component (e.g., intensity), a green component (e.g., intensity),contrast, brightness, and any other suitable color or image parameter tomatch the particular color element in the image to a known colorproperty of a reference color element in a reference color grid (e.g., acolor grid imaged under controlled environmental conditions, etc.). Inthis variation, once the image is thus normalized, the method 100 canimplement a parametric model to extrapolate a blood componentconcentration from the region of the image corresponding to the fluid inthe canister. This variation of the method 100 can be performed by theprocessing system described in Section 2 above, but can additionally oralternatively be implemented using any other suitable system.

In particular, Block S150′ recites: retrieving a true colorcharacteristic corresponding to the shade of red in the original imagebased on a position of the particular shade of red in the color grid inthe original image. Block S150′ can thus comprise implementing a methodor technique similar to Block S140′ described above to retrieve a realor known color value of a color element (or a subset of color elements)selected in the image in Block S130′. For example, Block S150′ canretrieve this color information from local memory on a mobile computingdevice (e.g., a tablet, a smartphone) executing the method 100 or from aremote color grid database.

Block S160′ recites: generating an adjusted image from the originalimage upon adjusting a color setting of the original image to align acolor characteristic of the shade of red in the adjusted image to thetrue color characteristic. Generally, once the reference color value forthe selected color element is collected, Block S160′ can includeadjusting at least one of: a red component (e.g., intensity), a bluecomponent (e.g., intensity), a green component (e.g., intensity),contrast, brightness, and any other suitable color or image parameter tomatch the color value of the selected color element in the image to thereference color value of the color element. For example, if Block S130′selects a color element with the address ((4,9), rbg(156,8,9)) but thereference color value of the color element in row 4, column 9 of thecolor grid is rgb(150, 0, 12), Block S160′ can include decreasing a redcomponent of the image, decrease a green component of the image, andincrease a blue component of the image until the color value of the(4,9) color element reaches rgb(150, 0, 12).

In another implementation, for two color elements selected in BlockS130′, Block S160′ can include implementing similar functionality asdescribed above in relation to Block S140′ to interpolate a target colorvalue between the real color values of the two selected color elements,and Block S160′ can then adjust the image color properties accordingly.

Furthermore, Block S160′ can include generating a fit parameter derivedfrom alignment of the first color characteristic in the adjusted imageto the first true color characteristic S162′; and providing anindication of suitability of the image, based upon the fit parameterS163′ (e.g., providing an indication at a display of a mobile computingdevice in communication with the computer system). In particular, thefit parameter can be derived from one or more functions fit between atrue color characteristic (e.g., red intensity, blue intensity, greenintensity) and a color characteristic derived from the original image,wherein the fit parameter can include one or more of: an r-value of aline fit, a parameter derived from outlier detection in relation to afitted function, and any other suitable fit parameter. This aspect ofBlock S160′ can thus be used as a quality control step that enables auser or other entity associated with the volume of fluid to be informedof unsuitable image data acquired of the canister and/or the color grid(e.g., in relation to scuffing of the color grid, marking of the colorgrid, or any other suitable image artifact that renders the imageunsuitable for processing).

Block S170′ recites: extracting a redness value from a region of theadjusted image, and Block S190′ of this variation of the method recites:correlating the redness value with a concentration of the bloodcomponent within fluid contained in the canister. Generally, once theimage is color normalized, Block S170′ and Block S190′ cooperate toimplement a parametric model or algorithm to convert the normalizedcolor of the selected region in the adjusted image—corresponding tofluid in the canister—into a blood component concentration, as describedin U.S. patent application Ser. No. 14/072,625. As such, Blocks S170′and S190′ can include selecting a region of the adjusted imagecorresponding to a portion of the canister, based upon a position of thecolor grid relative to the canister; determining a concentration of theblood component from a set of color parameters (e.g., red colorintensity, green color intensity, and blue color intensity) derived fromthe region of the adjusted image (e.g., based upon a parametric model,based upon template matching, etc.); and determining an amount of theblood component within the canister, based upon determining a volume offluid within the canister. Blocks S170′ and/or S190′ of the firstvariation of the method 100′ can, however, be implemented in any othersuitable manner.

Furthermore, as noted above, variations of the method 100′ can beadapted to process image data (or other data) derived from any othersuitable fluid receiver (e.g., canister, test strip, absorbent pad,surgical textile, sponge, fluid receiving bag, drape, cell salvagesystem, drain device, etc.) associated with or otherwise coupled to acolor grid (e.g., incorporated into a quick response code, incorporatedinto a barcode, incorporated into a rectilinear array, incorporated intoan axially symmetric array, etc.), wherein the fluid receiver isconfigured to receive (e.g., receive into a cavity, receive uponabsorption) a volume of fluid (e.g., urine, saline, ascites, bile,irrigant saliva, gastric fluid, mucus, pleural fluid, interstitialfluid, fecal matter, etc.). In particular, color elements of the colorgrid, as described above, can comprise any other suitable (e.g.,non-red) color(s) configured to facilitate image data normalization forenvironmental condition effects, and/or for extraction of relevant fluidcomponent characteristics (e.g., concentrations, purities, etc.) from avolume of fluid at the fluid receiver.

In one such variation, as shown in FIG. 6C, a method 100 b′ can include:within an original image of a fluid receiver, detecting a color grid inproximity to the fluid receiver S110 b′, the color grid comprising anarray of color elements, wherein each color element is associated withat least one of a set of colors; selecting a region of the originalimage corresponding to a portion of the fluid receiver, based upon aposition of the color grid relative to the fluid receiver S120 b′;retrieving a true color characteristic corresponding to a color in theoriginal image based upon a position of the color in the color grid inthe original image S150 b′; generating an adjusted image derived fromthe original image upon adjusting a color setting of the original imageto align a color characteristic of the color in the adjusted image tothe true color characteristic S160 b′; extracting a color value from aregion of the adjusted image S170 b′; and determining a concentration ofthe blood component within a volume of fluid at the fluid receiver,based upon the color value S190 b′.

As such, variations of the method 100′ can facilitate mitigation ofambient light effects (and other effects) in an environment of the fluidreceiver, in determining a concentration and/or an amount of a fluidcomponent within a volume of fluid received at the fluid receiver.

5. Second Variation

As shown in FIGS. 7A and 7B, a second variation of the method 100″includes: within an original image of a canister, detecting a color gridcoupled to the canister S110″, the color grid comprising an array ofcolor elements, wherein each color element is associated with at leastone of a set of colors in a red spectrum; retrieving a first true colorcharacteristic corresponding to a first shade of red in the set ofcolors captured in the original image based on a position of the firstshade of red in the color grid in the original image S150″; retrieving asecond true color characteristic corresponding to a second shade of redin the set of colors captured in the original image based on a positionof the second shade of red in the color grid in the original imageS152″; generating an adjusted image from the original image uponadjusting a color setting of the original image to align a first colorcharacteristic of the first shade of red in the adjusted image to thefirst true color characteristic and to align a second colorcharacteristic of the second shade of red in the adjusted image to thesecond true color characteristic S160″; extracting a redness value froma region of the adjusted image corresponding to a portion of thecanister based on a position of the color grid relative to the canisterS170″; and correlating the redness value with a concentration of theblood component within a volume of fluid contained in the canisterS190″.

Generally, in this variation, the method 100″ normalizes the image ofthe canister according to color values of one or more color elements inthe image and corresponding reference color values of the colorelements. In particular, rather than matching a color value of a colorelement in the color grid in the image to a color in a selected regionof the image corresponding to fluid in the canister, this variation ofthe method immediately color normalizes the image according to theimaged and reference color values of one or more color elements in thegrid, and then implements a parametric model to extrapolate a bloodcomponent concentration in the fluid in the imaged canister, asdescribed in relation to the processing system in Section 2 above.

In particular, in this variation, Blocks S150″ and S152″ can implementmethods and techniques similar to those of Blocks S140′ and S150′described above to retrieve true color characteristics corresponding toa first shade of red and a second shade of red in the set of distinctshades of red in the original image based on a position of the firstshade of red in the color grid in the original image. Alternatively,Block S150″ and S152″ can retrieve true color values of a red colorelement and a black color element in the color grid in the image of thecanister. However, Blocks S150″ and S152″ can retrieve any other truecolor value of one or more color elements in the color grid.

In this variation, Blocks S160″, S170″, and S190″ can then implementmethods or techniques similar to Blocks of the foregoing variation toadjust the image, to extract a redness value from a region of theadjusted image corresponding to fluid in the canister, and to correlatethe redness value with a concentration of the blood component within avolume of fluid contained in the canister.

Furthermore, as noted above, variations of the method 100″ can beadapted to process image data (or other data) derived from any othersuitable fluid receiver (e.g., canister, test strip, absorbent pad,surgical textile, sponge, fluid receiving bag, drape, cell salvagesystem, drain device, etc.) associated with or otherwise coupled to acolor grid (e.g., incorporated into a quick response code, incorporatedinto a barcode, incorporated into a rectilinear array, incorporated intoan axially symmetric array, etc.), wherein the fluid receiver isconfigured to receive (e.g., receive into a cavity, receive uponabsorption) a volume of fluid (e.g., urine, saline, ascites, bile,irrigant saliva, gastric fluid, mucus, pleural fluid, interstitialfluid, fecal matter, etc.). In particular, color elements of the colorgrid, as described above, can comprise any other suitable (e.g.,non-red) color(s) configured to facilitate image data normalization forenvironmental condition effects, and/or for extraction of relevant fluidcomponent characteristics (e.g., concentrations, purities, etc.) from avolume of fluid at the fluid receiver.

In one such variation, as shown in FIG. 7C, a method 100 b″ includes:within an original image of a fluid receiver, detecting a color gridcoupled to the fluid receiver S110 b″, the color grid comprising anarray of color elements, wherein each color element is associated withat least one of a set of colors; retrieving a first true colorcharacteristic corresponding to a color in the set of colors captured inthe original image based on a position of the first color in the colorgrid in the original image S150 b″; retrieving a second true colorcharacteristic corresponding to a second color in the set of colorscaptured in the original image based on a position of the second colorin the color grid in the original image S152 b″; generating an adjustedimage from the original image upon adjusting a color setting of theoriginal image to align a first color characteristic of the first colorin the adjusted image to the first true color characteristic and toalign a second color characteristic of the second color in the adjustedimage to the second true color characteristic S160 b″; extracting acolor value from a region of the adjusted image corresponding to aportion of the fluid receiver based on a position of the color gridrelative to the fluid receiver S170 b″; and correlating the color valuewith a concentration of the blood component within a volume of fluid atthe fluid receiver S190 b″.

As such, variations of the method 100″ can facilitate mitigation ofambient light effects (and other effects) in an environment of the fluidreceiver, in determining a concentration and/or an amount of a fluidcomponent within a volume of fluid received at the fluid receiver.

6. Noise Removal and Blood Component Amount

As shown in FIG. 3E, the method 100 can additionally include Block S50,which recites: removing image artifacts present in image data associatedwith at least one of the canister and the color grid, which functions toimprove outputs of subsequent blocks of the method 100. In particular,Block S50 can function to remove noise or other artifacts (e.g.,artifacts caused by air bubbles under the color grid, artifacts causedby debris between the color grid 300 and the canister, etc.) from theimage data of the region of interest for blood component analysis. Invariations, Block S50 can comprise implementing one or more of: amaximally stable extremal regions (MSER) algorithm, a Laplacian ofGaussian algorithm, a difference of Gaussians algorithm, and adeterminant of Hessian algorithm, and any other suitable algorithm toremove noise or other artifacts. In a specific example, as described inrelation to the processing system of Section 2 above, artifact removalin Block S50 can comprise implementing a maximally stable extremalregions (MSER) algorithm to determine an initial mask of substantiallyartifact-less subregions of the region of interest. Then, Block S50 cancomprise removing any pixels whose color value is significantlydifferent from a median color value of pixels of the initial mask, inorder to remove aberrations present after implementation of the MSERalgorithm. Variations of Block S50 can, however, be performed in anyother suitable manner.

As shown in FIG. 8, the method 100 can additionally include Block S192,which recites: within an image of a canister, identifying a referencemarker on the canister, selecting an area of the image based on thereference marker, correlating a portion of the selected area with afluid level within the canister, estimating a volume of fluid within thecanister based on the fluid level, and estimating a quantity of theblood component within the canister based on the estimated volume andthe concentration of the blood component. Alternatively, Block S192 cancomprise receiving information pertaining to the volume of fluid withinthe canister by an entity interacting with the system. In a firstvariation, the volume of fluid within the canister can be manually input(e.g., with keypad strokes, by speech, etc.) into an input module of acomputing device of the system. In an example of the first variation, aholistic blood loss management application executing at a mobilecomputing device (e.g., tablet computer, smartphone device, etc.) caninclude a user interface configured to receive an input indicative ofthe volume of fluid within the canister, wherein the input is providedby a physician, nurse, assistant, or technician present within anoperating room environment. In the example, the Block S192 can thus usethe input volume of fluid information in estimating a quantity of theblood component within the canister. However, in alternative variationsand examples, the quantity of the blood component within the canistercan be determined in any other suitable manner.

Additionally or alternatively, Block S192 can implement methods andtechniques described in U.S. patent application Ser. No. 14/072,625 todetect the volume of fluid in the canister and to calculate the totalvolume, mass, weight, or other method of the blood component in thecanister based on the total volume and the estimated blood componentconcentration in the canister.

7. Specific Example Pipeline

As shown in FIG. 9, in one specific example of a pipeline associatedwith the method(s) 100, 100 b, 100′, 100 b′, 100″, 100 b″ andimplementing an embodiment of the described system: software executingon a server 400, in communication with a mobile computing device 500that captures the image of the canister and color grid, can facilitatetransmission of image and canister type data to a processing module 420(e.g., computing subsystem) for estimation of the amount of the bloodcomponent within a fluid canister. The processing module 420 can includea first module 422 configured to perform image qualification algorithms(e.g., image processing algorithms, image conditioning algorithms,etc.), and configured to provide an output to a second module 424configured to extract information from a matrix barcode (e.g., QRcode-color grid hybrid, in relation to Block S110 above) captured in theimage. The information from the matrix barcode can then be passedthrough a transformation module 426, that implements a transformationfrom canonical space to image space (e.g., based upon a template of howthe matrix barcode should look in a known configuration), to a thirdmodule 428 for extraction of ambient light palette information (i.e., inrelation to Block S150 above) and a fourth module 430 for processing ofinformation related to a region of interest (i.e., in relation to BlockS120 above). Outputs of the third module 428 and the fourth module 430are then transmitted to an ambient light normalization module 432configured to generate a corrected image from the original image,wherein the corrected image is normalized in a manner that accounts forvariations in ambient light conditions. An output of the third module428 is also transmitted to a noise filtering module 434 configured tomitigate effects of noise in the image. Then, outputs of the ambientlight normalization module 432 and the noise filtering module 434 aretransmitted to a feature extraction module 436 (e.g., in relation toBlocks S140, S170, and S190 above) in order to retrieve an estimatedhemoglobin concentration associated with the region of the image andrepresentative of the hemoglobin concentration of fluid within thecanister. Finally, the estimated hemoglobin concentration and anestimated volume of fluid within the canister (e.g., in relation toBlock S192 above) are used to calculate a mass of hemoglobin within thecanister, which is then transmitted back to the server 400 (e.g., andeventually provided to an entity associated with a patient from whom theblood originated). Variations of the specific example can, however, beimplemented in any other suitable manner.

The systems and methods of the preferred embodiment can be embodiedand/or implemented at least in part as a machine configured to receive acomputer-readable medium storing computer-readable instructions. Theinstructions are preferably executed by computer-executable componentspreferably integrated with the application, applet, host, server,network, website, communication service, communication interface,hardware/firmware/software elements of a user computer or mobile device,or any suitable combination thereof. Other systems and methods of thepreferred embodiment can be embodied and/or implemented at least in partas a machine configured to receive a computer-readable medium storingcomputer-readable instructions. The instructions are preferably executedby computer-executable components preferably integrated bycomputer-executable components preferably integrated with apparatusesand networks of the type described above. The computer-readable mediumcan be stored in the cloud and/or on any suitable computer readablemedia such as RAMs, ROMs, flash memory, EEPROMs, optical devices (CD orDVD), hard drives, floppy drives, or any suitable device. Thecomputer-executable component is preferably a processor but any suitablededicated hardware device can (alternatively or additionally) executethe instructions.

The FIGURES illustrate the architecture, functionality and operation ofpossible implementations of systems, methods and computer programproducts according to preferred embodiments, example configurations, andvariations thereof. In this regard, each block in the flowchart or blockdiagrams may represent a module, segment, step, or portion of code,which comprises one or more executable instructions for implementing thespecified logical function(s). It should also be noted that, in somealternative implementations, the functions noted in the block can occurout of the order noted in the FIGS.. For example, two blocks shown insuccession may, in fact, be executed substantially concurrently, or theblocks may sometimes be executed in the reverse order, depending uponthe functionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts, or combinations of special purpose hardware andcomputer instructions.

As a person skilled in the art will recognize from the previous detaileddescription and from the figures and claims, modifications and changescan be made to the preferred embodiments of the invention withoutdeparting from the scope of this invention as defined in the followingclaims.

We claim:
 1. A method for assessing an concentration of a fluidcomponent within a container, comprising: at a computing system incommunication with an image acquisition device, receiving dataassociated with an image of the container generated by the imageacquisition device; from the image, detecting a color grid comprising anarray of color elements coupled to the container, wherein each colorelement of the color grid is associated with at least one of a set ofcolors; at the computing system, selecting a region of the imageassociated with the container; and at the computing system, determininga concentration of the fluid component from the region of the imagebased upon processing of information derived from the color grid.
 2. Themethod of claim 1, wherein detecting the color grid comprises: detectingcorner features of the color grid.
 3. The method of claim 2, whereindetecting the color grid further comprises: fitting a homography betweencanonical space and an image space associated with the image; with thehomography, transforming each of a set of regions, associated with acolor element of the color grid, from canonical space to the image space4. A method of claim 1, further comprising: at the computing system,determining a first true color characteristic corresponding to a firstcolor in the set of colors captured in the image, based upon a positionof the first shade of a color in the color grid.
 5. The method of claim4, wherein the first true color characteristic is determined from areference dataset of the color grid, and wherein generating the imagecomprises adjusting at least one component color setting of the image toalign the first color characteristic of the first true colorcharacteristic.
 6. The method of claim 5, further comprising generatinga fit parameter derived from alignment of the first color characteristicin the image to the first true color characteristic; and providing anindication of suitability of the image, based upon the fit parameter. 7.The method of claim 1, wherein determining the concentration of thefluid component from the region of the image comprises determining theconcentration from a set of color parameters derived from the region ofthe image.
 8. The method of claim 7, wherein determining theconcentration from the set of color parameters comprises transforming atleast one color component into the concentration representative of theregion.
 9. The method of claim 8, further comprising determining anamount of the fluid component within the container, the amount derivedfrom the concentration multiplied by the volume of fluid within thecontainer.
 10. A method for assessing the concentration of a fluidcomponent received at a fluid receiver, the method implemented at acomputer system and comprising: receiving data associated with an imageof the fluid receiver; from the image, detecting a color grid comprisingan array of color elements in proximity to the fluid receiver, whereineach color element of the color grid is associated with at least one ofa set of colors; selecting a region of the image corresponding to aportion of the fluid receiver; and determining the concentration of thefluid component from the region of the image, based upon processing ofinformation derived from the color grid.
 11. The method of claim 10,wherein receiving data associated with an image of the fluid receivercomprises receiving data associated with the image of at least one of: acanister, a test strip, an absorbent pad, a surgical textile, a sponge,a fluid receiving bag, a drape, a cell salvage system, and a draindevice.
 12. The method of claim 11, wherein detecting the color gridcomprises: detecting a positional feature of the color grid; determiningan identifier of the fluid receiver upon decoding information from thecolor grid; and retrieving a template of the color grid in canonicalspace upon reception of the identifier.
 13. The method of claim 12,further comprising: determining a first true color characteristiccorresponding to a first color in the set of colors captured in theimage, based upon a position of the first color in the color grid in theimage; and generating an adjusted image upon adjusting a color settingof the image to align a first color characteristic of the first color inthe adjusted image to the first true color characteristic; wherein thefirst true color characteristic is determined from a reference datasetof the color grid, and wherein generating the adjusted image comprisesadjusting at least one of a red component color setting, a greencomponent color setting, and a blue component color setting of the imageto align the first color characteristic of the first color in theadjusted image to the first true color characteristic.
 14. The method ofclaim 10, wherein determining the concentration of the fluid componentfrom the region of the adjusted image comprises at least one of 1)associating a color value of the region of the adjusted image with acolor element of the color grid, wherein the color element of the colorgrid is associated with a predetermined concentration of the fluidcomponent; and 2) generating a set of color parameters from the regionof the adjusted image, and determining the concentration of the fluidcomponent based upon a parametric model that receives the set of colorparameters as inputs.
 15. The method of claim 14, further comprisingremoving noise from the region of the adjusted image based upon analgorithm configured remove any pixels from the region whose color valueis significantly different from a median color value of pixels of theregion.
 16. The method of claim 11, wherein determining theconcentration of the fluid component from the region of the adjustedimage comprises determining the concentration from a set of colorparameters derived from the region of the adjusted image, the set ofcolor parameters including a red component, a green component, and ablue component.
 17. The method of claim 16, wherein determining aconcentration of the fluid component from the region comprisesdetermining the concentration of a bodily fluid component derived fromat least one of: urine, saline, ascites, bile, irrigant saliva, gastricfluid, mucus, pleural fluid, interstitial fluid, and fecal matter.
 18. Asystem for assessing an amount of a fluid component, the systemcomprising: a fluid receiver coupled to a color grid, the color gridcomprising an array of color elements in proximity to the fluidreceiver, wherein each color element of the color grid is associatedwith at least one of a set of colors; an image acquisition deviceconfigured to capture an image of the fluid receiver and the color gridwithin a window of view; and a computing system in communication withthe image acquisition device and comprising: a first module configuredto detect the color grid within the image; a second module configured todetermine a first true color characteristic corresponding to a firstcolor in the set of colors captured in the image, based upon a positionof the first color in the color grid in the image; a third moduleconfigured to generate an adjusted image upon adjusting a color settingof the image to align a first color characteristic of the first color inthe adjusted image to the first true color characteristic; a fourthmodule configured to determine a concentration of the fluid componentfrom a region of the adjusted image corresponding to a portion of thefluid receiver; and a fifth module configured to generate an analysisindicative of the amount of the fluid component at the fluid receiver,based upon the concentration of the fluid component and a volume offluid received at the fluid receiver.
 19. The system of claim 18,wherein the image acquisition device comprises a display configured torender fluid component amount information, derived from the analysis.20. The system of claim 19, wherein the fluid receiver comprises acanister configured to receive fluid including a blood component,wherein the color grid is integrated with a quick response codeincluding the array of color elements, and wherein the computing systemis configured to decode an identifier of the canister from the quickresponse code.
 21. The system of claim 18, wherein the fourth module ofthe computing system is configured to determine a hemoglobinconcentration associated with the region, upon processing a red colorcomponent, a green color component, and a blue color component of theregion with a parametric model, and wherein the fifth module of thecomputing system is configured to determine a hemoglobin mass associatedwith the volume of fluid received at the fluid receiver.